Papers with real-time interactions
Proactive Guidance of Multi-Turn Conversation in Industrial Search (2025.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced multi-turn conversation systems, emphasizing the need for proactive guidance to enhance users’ interactions. |
| Approach: | They propose a goal-adaptive supervised fine-tuning framework that generates proactive guidance for users to click for the next turn of the conversation. |
| Outcome: | The proposed framework achieves 86.10% accuracy in offline evaluation (+23.95% over baseline) and 25.28% CTR in online deployment (149.06% relative improvement). |
Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding (2025.findings-naacl)
Copied to clipboard
Sukmin Cho, Sangjin Choi, Taeho Hwang, Jeongyeon Seo, Soyeong Jeong, Huije Lee, Hoyun Song, Jong C. Park, Youngjin Kwon
| Challenge: | Existing methods for drafting and verifying tokens require significant fine-tuning or have inconsistent performance across tasks. |
| Approach: | They propose a lossless drafting approach that organizes various token sources into multiple databases in a hierarchical framework based on temporal locality. |
| Outcome: | The proposed method outperforms existing database drafting methods on Spec-Bench using 7B and 13B parameters. |
Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models (2024.emnlp-main)
Copied to clipboard
Xinrong Zhang, Yingfa Chen, Shengding Hu, Xu Han, Zihang Xu, Yuanwei Xu, Weilin Zhao, Maosong Sun, Zhiyuan Liu
| Challenge: | Large language models (LLMs) are increasingly permeating daily lives and require real-time interactions that mirror human conversations. |
| Approach: | They propose to use time-division-multiplexing to process queries and responses pseudo-simultaneously. |
| Outcome: | The proposed model can listen to users while generating output and adjust to provide instant feedback. |
How Much Do Robots Understand Rudeness? Challenges in Human-Robot Interaction (2024.lrec-main)
Copied to clipboard
| Challenge: | This paper examines the pressing need to understand and manage inappropriate language within the evolving human-robot interaction landscape. |
| Approach: | They propose to use data cleaning methods to identify inappropriate language in real-time interactions and evaluate natural language models for their proficiency in discerning rudeness. |
| Outcome: | The proposed methods identify and mitigate inappropriate language in real-time interactions and evaluate natural language models for their proficiency in discerning rudeness. |
DiSCo: Device-Server Collaborative LLM-based Text Streaming Services (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have introduced significant cost and quality of experience (QoE) challenges in serving millions of daily requests. |
| Approach: | They propose a device-server cooperative scheduler that optimizes users’ QoE by adaptively routing requests and migrating response generation between endpoints while maintaining cost constraints. |
| Outcome: | Evaluations on real-world workloads show that the proposed scheduler can reduce tail TTFT (11-52%) and mean TTTT (6-78%) while maintaining comparable QoE levels. |